Run the out-of-band management power-state task in a managed context to fix a connection leak - #14090
Open
nagaboinaramgopal wants to merge 1 commit into
Open
Conversation
… context The per-host power-state sync submitted to the background executor was a plain Runnable, so the database connection its DB work acquires on the worker thread was never released back to the pool. Over time one leaked connection per configured host per run exhausted the pool (HikariPool active reaching maxActive), and the management server stopped serving requests. Make PowerOperationTask a ManagedContextRunnable, like the poll task that submits it, so the managed context releases the connection when each run finishes. Fixes: apache#13382
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The out-of-band management power-state poll task submits a per-host power status task to a background executor. That task, PowerOperationTask, was a plain Runnable, while the poll task that submits it is a ManagedContextRunnable. Running the per-host work outside a managed context means the database connection its DB work acquires on the worker thread is never released back to the pool. Over time this leaks one connection per configured host on every run, and once the pool reaches maxActive the management server stops serving requests with "Connection is not available, request timed out".
This makes PowerOperationTask a ManagedContextRunnable, matching the poll task that submits it, so the managed context releases the connection when each run finishes.
Fixes: #13382
Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
N/A
How Has This Been Tested?
Added a unit test that runs the task and checks it is a ManagedContextRunnable and that running it still performs the power operation, so the per-host work now runs inside a managed context and the delegation is unchanged.
How did you try to break this feature and the system with this change?
The change only wraps the existing per-host work in a managed context and does not change what it does. The sibling poll task that submits this task already uses the same managed-context base, so the two now behave consistently.